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A multilevel image into which a color or a 
back-and-whi te image is converted is input, and a 
slightly indistinct binary image generating unit 
generates a slightly indistinct binary image that 
includes a slightly indistinct line pattern and does 
not include background noise. Additionally, a 
shape-preserved binary image generating unit generates 
a binary image that preserves the shape of a line pattern 
and includes background noise. These images are ANDed 
for each pixel, so that a binary image that preserves 
the shape of the line pattern and does not include the 
background noise is generated. 



IMAGE PROCESSING APPARATUS 



Background of the Invention 
Field of the Invention 

The present invention relates to an image 
processing apparatus. 

Description of the Related Art 

In recent years, paper form and document 
recognition techniques using a non-contact type image 
input device (OHR: Over Head Reader) have received 
widespread attention. 

Fig. 1 shows the external view of an OHR. 

The non-contact type image input device (OHR) is 
an image input device of a standing type shown in Fig. 
1, which uses a line or an area CCD as an image pickup 
device. Using the OHR allows a user to feel comfortable 
in operations such as an operation for filling in a paper 
form or a document while inputting an image, an operation 
for inputting an image while looking at a paper form 
or a document, etc., in comparison ■ with a contact type 
image input device such as a conventional image scanner . 

However, an image captured by an OHR (OHR image) 
suffers degradation in image quality, such as density 
unevenness, a shadow, image deformation, etc., in 



comparison with an image captured by a scanner (scanner 
image) . 

Fig. 2 shows the state of a grayscale (graylevel?) 
scanner image, whereas Fig. 3 shows the state of a 
grayscale OHR image. It is known from the OHR image shown 
in Fig. 3 that the degree of density unevenness is 
slightly high and character lines are more blurred 
although no shadow exists. 

Fig. 4 shows the state of an OHR image with a shadow. 
As shown in this figure, the OHR image has a shadow. 
Therefore, a binarization technique that overcomes 
density unevenness, a shadow, etc. is essential to use 
an OHR. 

To enable a recognition process with high accuracy 
for an OHR image, a binarization method for obtaining 
a stable line pattern is required for a shadow/density 
unevenness. Binarization using a predetermined 
threshold is insufficient, and local binarization such 
as Niblack binarization, etc. must be introduced. For 
Niblack local- binarization, refer to the following 
document . 

<t> . D. Trier, A. K. Jain: "Goal-Directed 
Evaluation of Binarization Methods, " IEEE Transactions 
on Pattern Analysis and Machine Intelligence, Vol. 17. 
NO. 12, pp. 1191-1201, 1995 



Niblack local binarization is a method performing 
binarization for each pixel by defining that a threshold 
value of each pixel is T=E+ka (E and a are respectively 
a density average and a standard deviation in the 
neighborhood of a target pixel, and k is a constant of 
approximately -0.4 to 0.4) . A square area of N*N pixels, 
which centers on a target pixel, is used in the 
neighborhood of the target pixel (N=7 or so is frequently 
used) . 

Furthermore, if Niblack local binarization is 
applied unchanged, spotted black-and-white noise 
occurs in a background area due to the phenomenon that 
all of pixels in the neighborhood of a target pixel have 
uniform density. Fig. 5 exemplifies an OHR image, 
whereas Fig. 6 shows a binary image that is obtained 
by performing Niblack local binarization for the OHR 
image shown in Fig. 5. 

As is known from Fig. 6, spotted black-and-white 
noise occurs in a background area (in case of N=7, and 
k=0.1) . 

Therefore, the spotted black-and-white 
background noise is eliminated by determining a 
4-concatenated component with an average edge intensity 
of outline pixels, which is equal to or smaller than 
a predetermined value, of a 4-concatenated component 



of black pixels of a binary image to be the background 
noise, and by removing the concatenated component. The 
4-concatenated component of black pixels is a maximum 
set of black pixels, which is obtained by sequentially 
concatenating black pixels adjacent left and right and 
up and down. Also there is an 8-concatenated component 
containing the 4-concatenated component. This is a 
maximum set of black pixels, which is obtained by 
sequentially concatenating black pixels adjacent in 
four diagonal directions containing left and right and 
up and down. Here, also the 8-concatenated component 
may be used. If a simple term "concatenated component" 
appears hereinafter, it indicates a 4- or 8-concatenated 
component. The outline pixels of a concatenated 
component is black pixels included in the concatenated 
component, and a white pixel being background exists 
in any of up and down or left and right of each of the 
black pixels. The average edge intensity of outline 
pixels is an average of the edge intensities of the 
outline pixels. The edge intensity of an outline pixel 
is an edge intensity obtained with a Sobel edge filter, 
etc . 

Figs. 7A and 7B explain the concept of Niblack 
local binarization . 

As shown in Fig. 7A, for example, a square area 



of 7x7 pixels centering on each of pixels of a grayscale 
image obtained from a color or a black-and-white image 
is recognized to be a process target. Assume that an 
average of the densities of black pixels within the 
square area is E, and a standard deviation from the 
average E of the densities of the pixels within the 
square area is a. A threshold value T for determining 
whether a pixel to be binarized is either black or white 
is obtained by an equation T=E+ko. Also assume that the 
density of the pixel to be binarized is g. This pixel 
is made black if g^T, and made white if g>T . According 
to such determination results, binary images are 
sequentially obtained by providing black or white data 
after being binarized as the density data of the target 
pixel (Fig. 7B) . 

Fig. 8 is a flowchart showing the flow of the 
Niblack local binarization process. Firstly, in step 
SI, a pixel to be processed is selected. In step S2, 
densities of pixels within a square area centering on 
the selected pixel are obtained. In step S3, an average 
E of the densities of the pixels within the square area, 
and a standard deviation a are calculated. In step S4, 
a threshold value T is obtained with the equation T=E+ka 
(k is typically a value between -0.4 and 0.4). Then, 
in step S5, it is determined whether or not the density 



of the selected pixel is equal to or larger than the 
threshold value T. If the result of the determination 
made in step S5 is "YES", the selected pixel is made 
black. If the result of the determination made in step 
S5 is "NO", the selected pixel is made white. In step 
S8, it is determined whether or not all pixels of the 
image to be binarized have been processed. If a pixel 
yet to be processed is left, the process goes back to 
step SI and the subsequent operations are repeated. If 
it is determined that all the pixels have been processed, 
the process is terminated. 

Fig. 9 explains a Sobel edge filter and edge 
intensity . 

If a concatenated component shown in (1) of Fig. 
9 exists, a shaded portion corresponds to outline pixels . 
For the outline pixels, a square area of 3x3 pixels, 
which is shown in (4) and centers on a pixel to be 
processed, is taken, and the densities of pixels within 
the square area and filters shown in (2) and (3) of Fig. 
9 are multiplied and added. Suppose that a vector 
component generated by the filter in (2) is S x , and a 
vector component generated by the filter in (3) is S y . 
In this case, a vector S=(S x ,S y ) is obtained for the 
central pixel of the square area as shown in (4) . The 
length of this vector, namely, V (S x 2 +S y 2 ) is the Sobel 
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edge intensity of the target pixel. 

The average edge intensity is an intensity 
acquired by obtaining such edge intensities for all of 
the outline pixels of the concatenated component in (1) , 
and by averaging the obtained edge intensities. 

Fig. 10 shows a result of eliminating background 
noise by removing 4-concatenated components whose 
average edge intensities are equal to or smaller than 
4 . 

As described above, a binary image of relatively 
high quality can be obtained with a conventional 
technique from an image having relatively good contrast 
shown in Fig. 5. 

The background noise elimination used by 
conventional techniques stably and properly operates 
for a character having good contrast, but not for an 
image having poor contrast between a background and a 
character, that is, an image including an extremely 
faint character as shown in Fig. 11. 

A result of executing Niblack local binarization 
(k=0.1) for the grayscale image of Fig. 11 is shown in 
Fig. 12. 

If the background noise elimination is executed 
with the average edge intensity of 4 or smaller in a 
similar manner as in the case of Fig. 10, also the 
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extremely faint character string is removed together 
although the background noise can be eliminated. 

If the average edge intensity at the time of 
background noise elimination is set to 2 or smaller so 
as to preserve the line patterns of the extremely faint 
character string, this results in a binary image from 
which the background noise cannot be eliminated 
completely. 

As described above, the conventional techniques 
using the background noise elimination that adopts the 
local binarization and an average edge intensity have 
a problem in that an extremely faint character string 
cannot be satisfactorily extracted without including 
background noise. 

Summary of the Invention 

An object of the present invention is to provide 
an image processing apparatus that appropriately 
detects a line pattern from an input image having poor 
contrast, and performs a binarization process. 

An image processing apparatus according to the 
present invention is an image processing apparatus, to 
which an image including a line pattern such as a 
character, a ruled line, a graphic, etc. is input, 
outputs a corresponding binary image. This apparatus 



comprises: a slightly indistinct binary image 
generating unit generating a first image that includes 
almost an entire line pattern although its shape is 
indistinct; a shape-preserved binary image generating 
unit generating a second image that includes almost the 
entire line pattern and also noise in a background area 
other than the neighborhood of the line pattern; and 
an image combining unit combining the first and the 
second images for each pixel, and generating a 
satisfactory binary image which includes almost the 
entire line pattern while preserving its shape and does 
not include the noise in the background area. 

The present invention can provide an image 
processing apparatus that can securely capture the shape 
of a line pattern by combining images having different 
natures (the first and the second images. Particularly, 
these images are binary images) for each pixel 
(particularly, AND combination for each pixel) while 
eliminating noise in a background area. With the 
conventional techniques, a line pattern becomes 
slightly indistinct if attempts are made to securely 
capture its shape, or spotted black-and-white noise 
occurs in a background area if attempts are made to make 
the shape of a line pattern distinct. Here, particularly, 
the first image is a binary image where the shape of 
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a line pattern is slightly indistinct, whereas the 
second image is a binary image where the shape of the 
line pattern is not indistinct, but spotted 
black-and-white noise occurs in a background area. 

According to one preferred embodiment of the 
present invention, the first and the second images are 
combined after background areas of the first and the 
second images are detected, and a background noise 
elimination process is performed. As a result, the image 
obtained by combining the images becomes superior in 
quality . 

According to a further preferred embodiment of the 
present invention, postprocessing is performed for a 
result of combining the first and the second images, 
thereby improving the quality of a final image. 



Brief Description of the Drawings 

Fig. 1 is an external view of an OHR; 

Fig. 2 shows the state of a grayscale scanner 

image ; 

Fig. 3 shows the state of a grayscale OHR image; 
Fig. 4 shows the state of an OHR image with a 
shadow; 

Fig. 5 exemplifies an OHR image; 

Fig. 6 shows a binary image obtained by performing 



Niblack local binarization for an OHR image; 

Figs. 7A and 7B explain the concept of Niblack 
local binarization; 

Fig. 8 is a flowchart showing the flow of the 
Niblack local binarization process; 

Fig. 9 explains a Sobel edge filter and edge 
intensity; 

Fig. 10 shows a result of eliminating background 
noise by removing 4-concatenated components whose 
average edge intensities are equal to or smaller than 
4; 

Fig. 11 exemplifies a grayscale image including 
an extremely faint character string; 

Fig. 12 shows the state where Niblack local 
binarization is performed without noise elimination; 

Fig. 13 shows a binary image obtained by 
performing, for the image of Fig. 11, Niblack local 
binaization with which pixels whose average edge 
intensities are 4.0 or smaller are removed; 

Fig. 14 shows a binary image obtained by 
performing, for the image of Fig. 11, Niblack local 
binarization with which pixels whose average edge 
intensities are 2.0 or smaller are removed; 

Fig. 15 exemplifies a first configuration of an 
image processing apparatus according to a preferred 



embodiment of the present invention; 

Fig. 16 shows a slightly indistinct binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string; 

Fig. 17 shows a shape-preserved binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string 
shown in Fig. 11; 

Fig. 18 shows a combined binary image obtained by 
ANDing the slightly indistinct binary image and the 
shape-preserved binary image for each pixel; 

Fig. 19 exemplifies a second configuration of an 
image processing apparatus according to a preferred 
embodiment of the present invention; 

Fig. 20 shows a slightly indistinct binary image 
with background noise, which is obtained from the 
grayscale image that is shown in Fig. 11 and includes 
the extremely faint character string; 

Fig. 21 shows a slightly indistinct binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string; 

Fig. 22 shows a shape-preserved binary image with 
neighboring noise, which is obtained from the grayscale 
image that is shown in Fig. 11 and includes the extremely 
faint character string; 
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Fig. 23 shows a shape-preserved binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string; 

Fig. 24 shows a combined binary image obtained by 
5 ANDing the slightly indistinct binary image and the 

shape-preserved binary image for each pixel; 

Figs. 25A and 25B are graphs representing the 
P number of black pixels against an average edge intensity 

ij3 of a line pattern; 

4j 

10 Fig. 26 exemplifies a more specific example of the 

v.! 

m 



preferred embodiment according to the present 
invention; 

Fig. 27 shows a slightly indistinct binary image 



fU with background noise, which is obtained from the 



Cj 15 grayscale image that is sown in Fig. 11 and includes 

the extremely faint character string; 

Fig. 28 is a graph representing a characteristic 
amount of 4-concatenated components of a slightly 
indistinct binary image with background noise, that is, 
20 a distribution of the number of black pixels to an 

average edge intensity of concatenated components; 

Fig. 29 shows a slightly indistinct binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string; 
25 Fig. 30 shows a shape-preserved binary image with 



neighboring noise, which is obtained from the grayscale 
image that is shown in Fig. 11 and includes the extremely 
faint character string; 

Fig. 31 is a graph representing a characteristic 
amount of 4-concatenated components of a 
shape-preserved binary image with neighboring noise, 
that is, a distribution of the number of black pixels 
to an average edge intensity of concatenated components ; 

Fig. 32 shows a shape-preserved binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string; 

Fig. 33 shows a combined binary image obtained by 
ANDing the slightly indistinct binary image and the 
shape-preserved binary image for each pixel; 

Fig. 34 is a flowchart showing a background noise 
elimination process; 

Fig. 35 shows the configuration of an information 
processing device required when the preferred 
embodiments of the present invention is implemented by 
a program; 

Fig. 36 explains a usage pattern of program data. 

Description of the Preferred Embodiments 

According to preferred embodiments of the present 
invention, a slightly indistinct binary image that 



includes almost an entire pattern although its shape 
is indistinct, and a shape-preserved binary image that 
includes the line pattern while preserving its shape 
and also includes background noise other than the 
neighborhood of the line pattern are generated by using 
a nature such that a distribution of the number of black 
pixels to an average edge intensity of concatenated 
components (a distribution of concatenated components 
when a graph is plotted by taking the number of black 
pixels and an average edge intensity of a concatenated 
component as axes) of a line pattern, and that of 
concatenated components of background noise overlap 
regarding a binary image that preserves the shape of 
a line pattern such as a character, a ruled line, a 
graphic, etc. and is resultant from local binarization, 
whereas these distributions are separate regarding a 
binary image whose line pattern is slightly made 
indistinct and which is resultant from local 
binarization, and these binary images are ANDed, so that 
a satisfactory image that includes almost the entire 
line pattern while preserving its shape and does not 
include the noise in the background area is generated. 

Fig. 15 exemplifies a first configuration of an 
image processing apparatus according to a preferred 
embodiment of the present invention. 
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The image processing apparatus having this 
exemplified configuration, to which a monochrome 
multilevel image (including an image into which a color 
image is converted) that includes a line pattern such 
5 as a character, a ruled line, a graphic, etc. is input, 

comprises a slightly indistinct binary image generating 
unit 10, a shape-preserved binary image generating unit 

O 11, and a binary image combining unit 12. When a binary 

■42 

yg image obtained by extracting a line pattern as black 

h 

jg 10 pixels is output, the slightly indistinct binary image 

^ generating unit 10 generates a slightly indistinct 

Fj I 

binary image that includes almost the entire lxne 
f~ pattern although its shape is indistinct, and does not 

FU include noise in a background area, the shape-preserved 

irof 

P 15 binary image generating unit 11 generates a 

La 

shape-preserved binary image that includes almost the 
entire line pattern while preserving its shape, and also 
includes noise in a background area other than the 
neighborhood of the line pattern, and the binary image 

20 combining unit 12 combines the slightly indistinct 

binary image and the shape-preserved binary image. As 
a result, a satisfactory binary image that includes 
almost the entire line pattern while preserving its 
shape and does not include the noise in the background 

25 is generated and output. 



Fig. 16 shows a slightly indistinct binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string. 

This binary image includes almost the entire line 
patterns although their shapes are indistinct, and does 
not include the noise in the background area. 

Fig. 17 shows a shape-preserved binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string. 

This binary image includes almost the entire line 
patterns while preserving their shapes, and also 
includes the noise in the background area other than 
the neighborhood of the line patterns. 

Fig. 18 shows a combined binary image obtained by 
ANDing the slightly indistinct binary image and the 
shape-preserved binary image for each pixel . This binary 
image is a satisfactory binary image that includes 
almost the entire line patterns while preserving their 
shapes and does not include the noise in the background 
area. It is proved from this preferred embodiment that 
an extremely faint character string can be 
satisfactorily extracted. 

In the above described and the following preferred 
embodiments, a slightly indistinct binary image is 
obtained by setting the value of a to a large positive 
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value, whereas a shape-preserved binary image is 
obtained by setting the value of a to a negative or a 
small value in the Niblack local binarization process. 

Accordingly, the algorithms of the processes 
performed by the slightly indistinct binary image 
generating unit 10 and the shape-preserved binary image 
generating unit 11, which are shown in Fig. 15, are the 
same Niblack local binarization process, and only 
difference exists between binarization threshold 
values in these algorithms . 

Fig. 19 exemplifies a second configuration of an 
image processing apparatus according to a preferred 
embodiment of the present invention. 

The image processing apparatus having this 
exemplified configuration, to which a multilevel image 
including a line pattern such as a character, a ruled 
line, a graphic, etc. is input, comprises a slightly 
indistinct binary image generating unit 10a, a 
shape-preserved binary image generating unit 11a, and 
a binary image combining unit 12. When a binary image 
obtained by extracting a line pattern as black pixels 
is output, the slightly indistinct binary image 
generating unit 10a generates a slightly indistinct 
binary image that includes almost an entire line pattern 
although its shape is indistinct, and does not include 



noise in a background area; the shape-preserved binary 
image generating unit 11a generates an image that 
includes almost the entire line pattern while preserving 
its shape, and also includes noise in a background area 
other than the neighborhood of the line pattern; and 
the binary image combining unit 12 combines the slightly 
indistinct binary image and the shape-preserved binary 
image. As a result, a satisfactory binary image that 
includes almost the entire line pattern while preserving 
its shape, and does not include the noise in the 
background area is generated and output. 

The slightly indistinct binary image generating 
unit 10a comprises a slightly indistinct binarizing unit 
13 and a background noise eliminating unit 15. The 
slightly indistinct binarizing unit 13 generates from 
an input multilevel image a slightly indistinct binary 
image with background noise, which includes almost an 
entire line pattern although its shape is indistinct 
and also includes noise in a background area, and the 
background noise eliminating unit 15 generates a 
slightly indistinct binary image that does not include 
the background noise from the slightly indistinct binary 
image with background noise . The shape-preserved binary 
image generating unit 11a comprises a shape-preserved 
binarizing unit 14, and a background noise eliminating 
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unit 15. The shape-preserved binarizing unit 14 
generates from an input multilevel image a 
shape-preserved binary image with neighboring noise, 
which includes almost an entire line pattern while 
5 preserving its shape and also includes noise in a 

background area containing the neighborhood of the line 
pattern. The background noise eliminating unit 15 
O generates a shape-preserved binary image, which 

yp includes almost the entire line pattern while preserving 

j= 10 its shape and also includes the noise in the background 

it area containing the neighborhood of the line pattern, 

as J 

by eliminating the background noise from the 

a 

M 5 shape-preserved binary image with neighboring noise. 

fij Note that the same algorithm of the background 

*=■■= 

□ 15 noise eliminating unit 15 is applicable to both the 



slightly indistinct binary image generating unit 10a 

and the shape-preserved binary image generating unit. 

A specific background noise elimination method will be 

described later. 
20 Fig. 20 shows a slightly indistinct binary image 

with background noise, which is obtained from the 

grayscale image that is shown in Fig. 11 and includes 

the extremely faint character string. 

This binary image includes almost the entire line 
25 patterns although their shapes are indistinct, and also 



L 



includes the noise in the background area. 

Fig. 21 shows a slightly indistinct binary image 
obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string. 

This binary image has a nature such that almost 
the entire line patterns are included although their 
shapes are indistinct, and the noise in the background 
area is not included. The reason why such a binary image 
can be generated is that a distribution of the number 
of black pixels to an average edge intensity of black 
pixel concatenated components including the line 
patterns of a slightly indistinct binary image with 
background noise is separate from a distribution of 
concatenated components including only the background 
noise of the image. 

Fig. 22 shows a shape-preserved binary image with 
neighboring noise, which is obtained from the grayscale 
image that is shown in Fig. 11 and includes the extremely 
faint character string. 

This binary image includes almost the entire line 
patterns while preserving their shapes, and also 
includes the noise in the background containing the 
neighborhood of the line patterns. 

Fig. 23 shows a shape-preserved binary image 
obtained from the grayscale image that is shown in Fig. 



11 and includes the extremely faint character string. 

This binary image has a nature such that almost 
the entire line patterns are included while their shapes 
are preserved, and also noise in a background area other 
than the neighborhood of the line patterns is included. 
The reason why also the noise in the background area 
is included is that a distribution of the number of black 
pixels to an average edge intensity of the black pixel 
concatenated components including the line patterns of 
the shape-preserved binary image with neighboring noise 
is not separate from a distribution of concatenated 
components including only the background noise. As a 
result, the background noise in the area where the 
distributions overlap cannot be eliminated, when the 
background noise is eliminated while preventing the line 
patterns from being erroneously eliminated. 

Fig. 24 shows a combined binary image obtained by 
ANDing the slightly indistinct binary image and the 
shape-preserved binary image for each pixel. 

This binary image is a satisfactory binary image 
that includes almost the entire line patterns while 
preserving their shapes, and does not include the noise 
in the background area. It is proved from this preferred 
embodiment that an extremely faint character string can 
be satisfactorily extracted. 



Any of the above described configurations 
utilizes a nature such that a distribution of the number 
of black pixels to an average edge intensity of 
concatenated components including line patterns is 
separate from that of concatenated components of 
background noise as shown in the graphs that represent 
the number of black pixels to an average edge intensity 
for line patterns and are shown in Figs. 25A and 25B, 
if a comparison is made for a binary image resultant 
from the local binarization between a distribution of 
a shape-preserved binary image (Fig. 25A) and that of 
a slightly indistinct binary image (Fig. 25B) by 
changing the binarization threshold value. 

A slightly indistinct binary image that includes 
almost the entire line patterns although their shapes 
are indistinct and does include the noise in the 
background area, and a shape-preserved binary image that 
includes almost the entire line patterns while 
preserving their shapes and also includes noise in a 
background area other than the neighborhood of the line 
patterns are generated, and the generated binary images 
are ANDed, whereby a satisfactory binary image that 
includes almost the entire line patterns while 
preserving their shapes and does not include the noise 
in the background area is generated. 
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This preferred embodiment allows an extremely 
faint character string to be satisfactorily extracted 
without including background noise, which cannot be 
implemented with the conventional techniques, as shown 
in Fig. 18 or 24 by effectively using a change in the 
characteristic distributions of concatenated 
components shown in Figs. 25A and 25B, and overcomes 
the problems caused by the conventional techniques . 

Fig. 26 exemplifies a more specific configuration 
of the preferred embodiment according to the present 
invention . 

The image processing apparatus shown in Fig. 26, 
to which a multilevel image including a line pattern 
such as a character, a ruled line, a graphic, etc. is 
input, comprises a slightly indistinct binary image 
generating unit 10a, a shape-preserved binary image 
generating unit 11a, and a binary image combining unit 
12. This apparatus outputs a binary image obtained by 
extracting a line pattern as black pixels. The slightly 
indistinct binary image generating unit 10a generates 
a slightly indistinct binary image that includes almost 
an entire line pattern although its shape is indistinct, 
and does not include noise in a background area; the 
shape-preserved binary image generating unit 11a 
generates a shape-preserved binary image that includes 



almost the entire line pattern while preserving its 
shape, and also includes noise in a background area other 
than the neighborhood of the line pattern; and the binary 
image combining unit 12 combines the slightly indistinct 
binary image and the shape-preserved binary image. As 
a result, a satisfactory binary image that includes 
almost the entire line pattern while preserving its 
shape and does not include the noise in the background 
area is generated and output. 

The slightly indistinct binary image generating 
unit 10a comprises a slightly indistinct binarizing unit 
13 and a background noise eliminating unit 15. The 
slightly indistinct binarizing unit 13 obtains a binary 
image with Niblack local binarization by defining that 
an average of densities of pixels in an area of 7x7 pixels 
centering on a target pixel is E, and a standard 
deviation is o, and by setting T=E+0 . 3a as a binarization 
threshold value for each pixel. If the value of k in 
the equation T=E+ka is a large value of approximately 
0.3, a slightly indistinct binary image with background 
noise, which includes almost an entire line pattern 
although its shape is indistinct and also includes noise 
in a background area, can be generated from an input 
multilevel image. 

The shape-preserved binary image generating unit 



11a comprises a shape-preserved binarizing unit 14, and 
a background noise eliminating unit 15. The 
shape-preserved binarizing unit 14 obtains a binary 
image with Niblack local binarization by defining that 
an average of densities of pixels in an area of 7x7 pixels 
centering on a target pixel is E, and a standard 
deviation is a, and by setting T=E+0 . la as a binarization 
threshold value for each pixel. If the value of k in 
the eguation T=E+ko is a small value of approximately 
0.1, a shape-preserved binary image with neighboring 
noise, which includes almost an entire line pattern 
while preserving its shape and also includes noise in 
a background area other than the neighborhood of the 
line pattern, can be generated from an input multilevel 
image . 

The background noise eliminating unit 15 
generates a binary image from the slightly indistinct 
binary image with background noise and the 
shape-preserved binary image with neighboring noise by 
eliminating the background noise. 

In this preferred embodiment, the background 
noise eliminating unit is implemented by the following 
method explained with a graph. 

Fig. 27 shows a slightly indistinct binary image 
with background noise, which is obtained from the 
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grayscale image that is shown in Fig. 11 and includes 
the extremely faint character string. 

This binary image includes almost the entire line 
patterns although their shapes are indistinct and also 
includes the noise in the background area. 

Fig. 28 is a graph representing a characteristic 
amount of 4-concatenated components of a slightly 
indistinct binary image with background noise, that is, 
a distribution of the number of black pixels of 
concatenated components to the average edge intensity 
of outline pixels. A distribution of concatenated 
components of background noise is clearly separate from 
that of concatenated components including line patterns. 
The reason is that the background noise in the 
neighborhood of the line patterns is concatenated with 
the line patterns, since the image is made indistinct. 
Accordingly, the background noise eliminating unit 
removes the concatenated components belonging to an area 
A, so that only the concatenated components including 
the line patterns can be extracted. A condition of 
belonging to the area A in this preferred embodiment 
is that the number of black pixels is equal to or smaller 
than 13, or an average edge intensity is equal to or 
smaller than 2.2, or the average edge intensity is equal 
to or smaller than (3.2- (the number of black 
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pixels-15) /65) . 

The background noise eliminating unit eliminates 
the background noise from the slightly indistinct binary 
image with background noise by using the condition to 
5 be satisfied for belonging to the area A, so that the 

slightly indistinct binary image that includes almost 
the entire line patterns although their shapes are 
indistinct and does not include the noise in the 
background area can be generated. 
10 Fig. 29 shows a slightly indistinct binary image 

obtained from the grayscale image that is shown in Fig. 
11 and includes the extremely faint character string. 

This binary image has a nature such that almost 
the entire line patterns are included although their 



q 15 shapes are indistinct, and the noise in the background 

area is not included. The reason why such a binary image 
can be generated is that a characteristic amount of the 
number of black pixels to the average edge intensity 
of black pixel concatenated components including the 
20 line patterns of a slightly indistinct binary image with 

background noise is separate from a distribution of the 
number of black pixels to the average edge' intensity 
of concatenated components including only the 
background noise. 
25 The shape-preserved binary image generating unit 



comprises a shape-preserved binarizing unit, and a 
background noise eliminating unit. The shape-preserved 
binarizing unit obtains a binary image with Niblack 
local binarization by defining that an average of 
densities of pixels within an area of 7x7 pixels 
centering on a target pixel is E, and a standard 
deviation is o, and by setting T=E+0 . la as a binarization 
threshold value for each pixel. If the value of k in 
the equation T=E+ka is a small value of approximately 
0.1, a shape-preserved binary image with neighboring 
noise, which includes almost an entire line pattern 
while preserving its shape and also includes noise in 
a background area containing also the neighborhood of 
the line pattern, can be generated from an input 
multilevel image. 

Fig. 30 shows a shape-preserved binary image with 
neighboring noise, which is obtained from the grayscale 
image that is shown in Fig. 11 and includes the extremely 
faint character string. 

This binary image includes almost the entire line 
patterns while preserving their shapes, and also 
includes the noise in the background area other than 
the neighborhood of the line patterns. 

Fig. 31 is a graph representing a characteristic 
amount of 4-concatenated components of a 
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shape-preserved binary image with neighboring noise, 
that is, a distribution of the number of black pixels 
of the concatenated components to the average edge 
intensity of outline pixels. 

A distribution of concatenated components of 
background noise and that of concatenated components 
including the line patterns overlap. The reason is that 
the background noise in the neighborhood of the line 
patterns is not concatenated with the line patterns, 
since the image is not indistinct. Therefore, the 
background noise eliminating unit removes the 
concatenated components belonging to an area B, so that 
a shape-preserved binary image that includes almost the 
entire line patterns while preserving their shapes and 
also includes the noise in the background area other 
than the neighborhood of the line patterns can be 
generated. A condition of belonging to the area B in 
this preferred embodiment is that the number of black 
pixels is equal to or smaller than 12 and the average 
edge intensity is equal to or smaller than 4, or the 
number of black pixels is equal to or smaller than 60 
and the average edge intensity is equal to or smaller 
than 2.8. 

The background noise eliminating unit eliminates 
the background noise from the shape-preserved binary 
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image with neighboring noise by using the above 
described condition to be satisfied for belonging to 
the area B, so that a shape-preserved binary image that 
includes almost the entire line patterns while 
5 preserving their shapes and also includes the noise in 

the background area other than the neighborhood of the 
line patterns can be generated. 

£3 Fig. 32 shows a shape-preserved binary image 

'42 

y= obtained from the grayscale image that is shown in Fig. 

y"1 

10 11 and includes the extremely faint character string. 

^2 This binary image has a nature such that almost 

the entire line patterns are included while their shapes 

a 

r= are preserved, and also the noise in the background area 

fjj other than the neighborhood of the line patterns is 

03 

ri 15 included. The reason why the noise in the background 

is included is that the background noise in the area 
where distributions of the number of black pixels to 
the average edge intensity of concatenated components 
including the line patterns and the background noise 

20 overlap cannot be eliminated, when the concatenated 

components of the background noise are removed from the 
shape-preserved binary image with neighboring noise. 

Fig. 33 shows a combined binary image obtained by 
ANDing the slightly indistinct binary image and the 

25 shape-preserved binary image for each pixel. 



This binary image is a satisfactory binary image 
that includes almost the entire line patterns while 
preserving their shapes and does not include the noise 
in the background area. It is proved from this preferred 
embodiment that an extremely faint character string can 
be satisfactorily extracted. 

A slightly indistinct binary image that includes 
almost the entire line patterns although their shapes 
are indistinct and does not include the noise in the 
background area, and a shape-preserved binary image that 
includes almost the entire line patterns while 
preserving their shapes and also includes the noise in 
the background area other than the neighborhood of the 
line patterns are generated by using a nature such that 
a distribution of the number of black pixels to the 
average edge intensity of concatenated components 
including the line patterns, and that of concatenated 
components of the background noise overlap regarding 
a binary image resultant from the local binarization, 
in which the shapes of the line patterns are preserved, 
whereas the distribution of the number of black pixels 
to the average edge intensity of concatenated components 
including the line patterns is separate from that of 
concatenated components of the background noise 
regarding a binary image resultant from the local 



binarization, in which the shapes of the line patterns 
are made indistinct. The generated binary images are 
ANDed for each pixel, so that a satisfactory binary image 
that includes almost the entire line patterns while 
preserving their shapes and does not include the noise 
in the background area is generated. 

Fig. 34 is a flowchart showing the above described 
background noise elimination process. 

Firstly, in step S10, the number of black pixels 
and the average edge intensity of each of concatenated 
components are obtained. Then, in step Sll, the number 
of black pixels and the average edge intensity of one 
of the concatenated components are selected as a 
concatenated component to be processed. In step S12, 
it is determined whether or not the number of black 
pixels is within a predetermined range. If the result 
of the determination is "NO", the process goes to step 
S15. If the result of the determination is "YES", it 
is further determined whether or not the average edge 
intensity is within a predetermined range in step S13. 
If the result of the determination is "NO" in step S13, 
the process goes to step S15. If the result of the 
determination is "YES" in step S13, the concatenated 
component to be processed is removed in step S14. The 
process then goes to step S15. In step S15, it is 



determined whether or not all the concatenated 
components have been processed. If a concatenated 
component yet to be processed is left, the process goes 
back to step Sll. If it is determined that all the 
concatenated components have been processed, the 
process is terminated. 

The above described example refers to the method 
introduced by the preferred embodiments according to 
the present invention as the background noise 
elimination. However, other methods such as the Yanowitz 
and Bruckstein's method, etc. are available. Or, for 
example, the method recited by Japanese Patent 
Application NO. 11-335495, which is a previous 
application of the present applicant, may be available. 
This application recites, as a background determination 
method, a method with which a target pixel is determined 
to be background if a standard deviation a of the 
densities of pixels within a neighboring area of the 
target pixel is smaller than a threshold value omin, 
or a target pixel is determined to be background if an 
average density difference Ag is smaller than a 
predetermined threshold value Agmin when the average 
density difference Ag={an average density of white 
pixels within a neighboring area - an average density 
of black pixels within the neighboring area} is defined. 



With this method, a pixel determined to be background 
is corrected to be white, so that spotted 
black-and-white noise is eliminated. Here, the average 
density difference is a difference between the average 
densities of the sets of white and black pixels in the 
case where neighboring pixels of a target pixel are 
tentatively binarized with the binarization threshold 
value of the target pixel. 

Additionally, as in the above described preferred 
embodiment, postprocessing may be further performed for 
an image to reduce noise after a slightly indistinct 
binary image and a shape-preserved binary image are 
ANDed . Namely, the background noise eliminating unit 
15 shown in Fig. 26 determines a concatenated component 
to be eliminated as background noise by judging whether 
or not the concatenated component is within a 
predetermined range specified by the number of black 
pixels and an average edge intensity, as shown in the 
graphs of Figs. 28 and 31. However, this is not 
sufficient for all cases. That is, background noise can 
possibly remain even after the AND process. 

Therefore, as described above, the method recited 
by Japanese Patent Application No. 11-225495, or the 
Yanowitz and Brucks tein ' s postprocessing, which is also 
available as the background noise eliminating unit, is 
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performed for a binary image after being ANDed. 
Especially, the Yanowitz and Bruckstein's 
postprocessing is described as the Postprocessing step 
in the above described paper. 
5 As described above, a binary image of higher 

quality can be obtained by using the method of the 
previous application or the Yanowitz and Bruckstein's 
r=| postprocessing in combination with the improvement in 

the quality of a binary image, which is implemented by 
10 the AND process of the preferred embodiments. 

^ Fig. 35 shows the configuration of an information 

42 

Pi processing device required when any of the preferred 

j=& embodiments according to the present invention is 

nj implemented by a program. 

03 

f~ 15 In the information processing device, a CPU 21 

id 

r& copies a program into a memory 22 via an external storage 

device 25 such as a hard disk, etc., or a medium driving 
device 26 reading a portable storage medium 29 (e.g., 
CD-ROM, floppy disk, DVD, etc.), which is connected by 

20 a bus 28, and executes the program. 

An input device 23 is configured by a keyboard, 
a mouse, a tablet, an OHR, etc., which a user of the 
information processing device utilizes to issue a 
command to the CPU 21, or to capture an image. For example, 

25 it is also possible to set the value of o in the Niblack 



binarization process, or to specify the range of the 
number of black pixels or an average edge intensity in 
the background noise elimination process. An output 
device 24 is configured by a display, a printer, etc., 
and is used to present a process result to a user, or 
to print a binary image. 

A network connecting device 27 is arranged to 
exchange data via a network, and can be used to read 
an image from an OHR apart from the information 
processing device, or to download a program for 
executing the processes of the preferred embodiments. 

Fig. 36 explains a usage pattern of program data. 

A program according to the preferred embodiments 
is stored in a memory 35 such as a RAM, a hard disk, 
etc. of an information processing device 31, and used. 
However, the program stored onto a CD-ROM, a floppy disk 

32, etc. may be loaded and used. Otherwise, program data 
may be downloaded from a program provider 30 via a line 

33, and used in the information processing device 31. 
Or, the program possessed by the program provider 30 
may be executed under a network environment without 
being downloaded. 

As described above, according to the present 
invention, a change in characteristic distributions of 
concatenated components is effectively used, whereby 
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an extremely faint character string within a multilevel 
image can be extracted without including background 
noise . 
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